When tomorrow comes: A prospective risk assessment of a future artificial general intelligence-based uncrewed combat aerial vehicle system.

There are concerns that Artificial General Intelligence (AGI) could pose an existential threat to humanity; however, as AGI does not yet exist it is difficult to prospectively identify risks and develop requisite controls. We applied the Work Domain Analysis Broken Nodes (WDA-BN) and Event Analysis of Systemic Teamwork-Broken Links (EAST-BL) methods to identify potential risks in a future 'envisioned world' AGI-based uncrewed combat aerial vehicle system. The findings suggest five main categories of risk in this context: sub-optimal performance risks, goal alignment risks, super-intelligence risks, over-control risks, and enfeeblement risks. Two of these categories, goal alignment risks and super-intelligence risks, have not previously been encountered or dealt with in conventional safety management systems. Whereas most of the identified sub-optimal performance risks can be managed through existing defence design lifecycle processes, we propose that work is required to develop controls to manage the other risks identified. These include controls on AGI developers, controls within the AGI itself, and broader sociotechnical system controls.

Identifying risk controls for future advanced brain-computer interfaces: A prospective risk assessment approach using work domain analysis.

Brain-computer interface (BCI) technologies are progressing rapidly and may eventually be implemented widely within society, yet their risks have arguably not yet been comprehensively identified, nor understood. This study analysed an anticipated invasive BCI system lifecycle to identify the individual, organisational, and societal risks associated with BCIs, and controls that could be used to mitigate or eliminate these risks. A BCI system lifecycle work domain analysis model was developed and validated with 10 subject matter experts. The model was subsequently used to undertake a systems thinking-based risk assessment approach to identify risks that could emerge when functions are either undertaken sub-optimally or not undertaken at all. Eighteen broad risk themes were identified that could negatively impact the BCI system lifecycle in a variety of unique ways, while a larger number of controls for these risks were also identified. The most concerning risks included inadequate regulation of BCI technologies and inadequate training of BCI stakeholders, such as users and clinicians. In addition to specifying a practical set of risk controls to inform BCI device design, manufacture, adoption, and utilisation, the results demonstrate the complexity involved in managing BCI risks and suggests that a system-wide coordinated response is required. Future research is required to evaluate the comprehensiveness of the identified risks and the practicality of implementing the risk controls.

Clear and present danger? Applying ecological interface design to develop an aviation risk management interface.

Ecological Interface Design (EID) is a framework for developing dynamic interfaces that support operators to understand and take appropriate actions within highly-complex systems. This two-part study involved the development and evaluation of a novel EID-based static aviation risk management display. A within-subjects survey-based experiment employed measures of decision accuracy, situation awareness, user workload, usability, and user perceptions. Results from 23 participants showed that when compared to a risk interface utilised by a large aviation company, the EID display achieved higher usability and lower workload ratings with large effect sizes, with no differences in decision accuracy and situation awareness. The findings provide evidence that the EID framework can improve upon static traditional displays. Research contributions also include a novel model of an aviation ground operations system and an example application of EID to the development of a static display. Further research is necessary to identify the extent of the applications and benefits to static displays.